1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method, a storage medium, and a program, in particular to an image processing apparatus and an image processing method, a storage medium, and a program in which an lod (Level Of Detail) is calculated by means of an apparatus with a simple structure.
2. Description of Related Art
In a field of computer graphics, there has been a technique referred to as texture mapping. When rendering a three dimension graphic (model), a prepared two dimension image (hereinafter referred to as texture) is pasted on a surface of the model so as to generate an image with high texture quality.
Referring now to FIG. 1 and FIG. 2, a basic principle of the texture mapping is briefly described below.
XY coordinates of FIG. 1A are coordinates in which the model is mapped. The texture is to be pasted on the model.
A model to be rendered is formed by assembling triangle polygons as shown in FIG. 1A. Vertices A, B, C of each polygon forming the model are provided with coordinate values (Sn, Tn, Qn, where n=1, 2, 3) of three-dimensional coordinates which are hereinafter referred to as texture coordinates (not shown in Figs. below).
Coordinate values (s, t, q) of a point D located in a inner field of the polygon are obtained by linearly interpolating texture coordinate values of points A, B, C. The coordinates (s, t) are homogeneous coordinates (s, t) of texture showing a pasted image pattern. A homogeneous term q is what is called an enlargement/reduction ratio.
Further, texture coordinate values such as the above-mentioned (s, t, q), (Sn, Tn, Qn) are provided for individual polygons which form a model to be rendered, and are variables. Vertex coordinate values (Sn, Tn, Qn, where n=1, 2, 3) of a polygon of the texture coordinates correspond to XY coordinate values (X, Y) and the texture coordinate values (s, t, q) of pixels within the polygon correspond to XY coordinate values (x, y).
UV coordinates of FIG. 1B are two-dimensional coordinates of texture pasted to the polygon of a model to be rendered. UV coordinate values (u, v) become (Sn/Qn, Tn/Qn) obtained by multiplying the homogeneous coordinates (Sn, Tn) of the polygon by a homogeneous term Q. Texture of FIG. 1B is pasted to the polygon in such a way that points A′, B′, C′, D′ correspond to the points A, B, C, D of the polygon mapped in the XY coordinates.
FIG. 2B shows how a line element dx (FIG. 2A), in the X-axis direction, of a unit pixel forming the XY coordinates corresponds to a displacement at UV coordinates. The line element dx at the XY coordinates corresponds to a displacement of du/dx in the U-axis direction at the UV coordinates and to a displacement of dv/dx in the V-axis direction. In other words, the du/dx and the dv/dx are respectively a displacement of u and a variation of v at UV coordinates when varying by (dx) at the XY coordinates. A displacement on at the UV coordinates corresponding to a line element dy (not shown) in the Y axis direction is similar to the above.
While, MIPMAP (Multum in parvo mapping) filtering is known as a method for obtaining a high resolution image when mapping texture. The MIPMAP filtering is described in Advanced Animation and Rendering Techniques (page.140) published by ADDISON WESLEY, for example.
As shown in FIG. 3 the MIPMAP filtering prepares a plurality of filtered texture data (original image, 1/2 image, 1/4 image, 1/8 image) respectively corresponding to a plurality of different reduction ratios (1/1, 1/2, 1/4, 1/8, for example, in FIG. 3) and selectively utilizes an optimal texture data corresponding to a reduction ratio of each pixel, thereby controlling an aliasing effect caused by information deletion when compressing an image so as to obtain a high resolution image.
FIG. 4 shows an example utilizing a texture mapping apparatus 1 for MIPMAP filtering.
A texture buffer 3 memorizes texture data carried out with a plurality of filtering processes each corresponding to a plurality of different reduction ratios as shown in FIG. 3
The texture mapping apparatus 1 calculates an lod (Level Of Detail) representing a reduction ratio of each pixel of a polygon. The texture mapping apparatus 1 reads an image, out of the texture buffer 3, corresponding to the calculated lod and outputs the image to a display buffer 4 so as to store it therein. An image based on the stored contents in the display buffer 4 is displayed on a display unit (not shown).
Below, operation of the texture mapping apparatus 1 is described with reference to a flow chart of FIG. 5.
In step S1, the texture mapping apparatus 1 inputs (s1, t1, q1), (s2, t2, q2), and (s3, t3, q3) indicating homogeneous coordinates and a homogeneous term with respect to each vertex of the polygon (FIG. 1A).
Then in step S2, the texture mapping apparatus 1 obtains (s, t, q) indicating homogeneous coordinates and a homogeneous term of each pixel within the polygon by linearly interpolating (s1, t1, q1), (s2, t2, q2), and (s3, t3, q3) of each inputted vertex.
In step S3, the texture mapping apparatus 1 calculates an lod of each pixel based on (s, t, q) of the each pixel within the polygon by means of a built-in lod calculating apparatus 2.
Here, as shown by equation (1), an lod may be represented by a logarithm, having a base of 2, of n if a reduction ratio of each pixel (s, t, q) is 1/n. Thus, lod's become 0, 1, 2, 3, . . . if reduction ratios are 1/1, 1/2, 1/4, 1/8, . . . respectively.Lod=log2(n)  (1)
The n of the reduction ratio (1/n) can be obtained by equation (2).
                    n        =                  MAX          (                      |                                          ⅆ                u                                            ⅆ                x                                      |            ,            |                                          ⅆ                v                                            ⅆ                x                                      |            ,            |                                          ⅆ                u                                            ⅆ                y                                      |            ,            |                                          ⅆ                v                                            ⅆ                y                                      |                    )                                    (        2        )            
The du/dx and the dv/dx in equation (2) are respectively a displacement of u and a variation of v at the UV coordinates when varying by (dx) at the XY coordinates (as shown in FIG. 2B); the du/dy and the dv/dy are respectively a displacement of u and a variation of v at the UV coordinates when varying by (dy) at the XY coordinates; and these are calculated according to equation (3). Thus, n is obtained based on equation (4).
                                                                                          ⅆ                  u                                                  ⅆ                  x                                            =                                                USIZE                  ×                                                            ⅆ                      S                                                              ⅆ                      x                                                        ×                                      1                    Q                                                  -                                  USIZE                  ×                                                            ⅆ                      Q                                                              ⅆ                      x                                                        ×                                      1                    Q                                    ×                                      S                    Q                                                                                                                                                            ⅆ                  v                                                  ⅆ                  x                                            =                                                VSIZE                  ×                                      dT                    Qx                                    ×                                      1                    Q                                                  -                                  VSIZE                  ×                                                            ⅆ                      Q                                                              ⅆ                      x                                                        ×                                      1                    Q                                    ×                                      T                    Q                                                                                                                                                            ⅆ                  u                                                  ⅆ                  y                                            =                                                USIZE                  ×                                                            ⅆ                      S                                                              ⅆ                      y                                                        ×                                      1                    Q                                                  -                                  USIZE                  ×                                                            ⅆ                      Q                                                              ⅆ                      y                                                        ×                                      1                    Q                                    ×                                      S                    Q                                                                                                                                                            ⅆ                  v                                                  ⅆ                  y                                            =                                                VSIZE                  ×                                                            ⅆ                      T                                                              ⅆ                      y                                                        ×                                      1                    Q                                                  -                                  VSIZE                  ×                                                            ⅆ                      Q                                                              ⅆ                      y                                                        ×                                      1                    Q                                    ×                                      T                    Q                                                                                                          (        3        )                                n        =                  MAX          (                      |                                          USIZE                ×                                  (                                                                                                              ⅆ                          S                                                                          ⅆ                          x                                                                    ×                      Q                                        -                                                                                            ⅆ                          Q                                                                          ⅆ                          x                                                                    ×                      S                                                        )                                                            Q                2                                      |                                                                                     ⁢                              |                                                      VSIZE                    ×                                          (                                                                                                                                  ⅆ                              T                                                                                      ⅆ                              x                                                                                ×                          Q                                                -                                                                                                            ⅆ                              Q                                                                                      ⅆ                              x                                                                                ×                          T                                                                    )                                                                            Q                    2                                                  |                                                                         |                                                                  USIZE                        ×                                                  (                                                                                                                                                      ⅆ                                  S                                                                                                  ⅆ                                  y                                                                                            ×                              Q                                                        -                                                                                                                            ⅆ                                  Q                                                                                                  ⅆ                                  y                                                                                            ×                              S                                                                                )                                                                                            Q                        2                                                              |                                                                                  ⁢                                                                                         |                                                                              VSIZE                            ×                                                          (                                                                                                                                                                          ⅆ                                      T                                                                                                              ⅆ                                      y                                                                                                        ×                                  Q                                                                -                                                                                                                                            ⅆ                                      Q                                                                                                              ⅆ                                      y                                                                                                        ×                                  T                                                                                            )                                                                                                            Q                            2                                                                          |                                            )                                                                                                                              (        4        )            
In equations (3) and (4), dS/dx, dT/dx, and dQ/dx represent differences of (S, T, Q) per pixel in an X direction, and dS/dy, dT/dy, and dQ/dy represent differences of (S, T, Q) per pixel in a Y direction. USIZE represents a width (length in a U direction) of the texture and VSIZE represents a height (length in a V direction) of the texture.
Thus, the lod is calculated upon operation of equation (5) where equation (1) is substituted with equation (4).
                    Lod        =                                            log              ⁢                                                                    2                    ⁢                      (                          MAX              (                                                                                          |                                                                        USIZE                          ×                                                      (                                                                                                                                                                ⅆ                                    S                                                                                                        ⅆ                                    x                                                                                                  ×                                Q                                                            -                                                                                                                                    ⅆ                                    Q                                                                                                        ⅆ                                    x                                                                                                  ×                                S                                                                                      )                                                                                                    Q                          2                                                                                                                                              ︸                    A                                                  ⁢                                                                  ⁢                                                      |                                                                  VSIZE                        ×                                                  (                                                                                                                                                      ⅆ                                  T                                                                                                  ⅆ                                  x                                                                                            ×                              Q                                                        -                                                                                                                            ⅆ                                  Q                                                                                                  ⅆ                                  x                                                                                            ×                              T                                                                                )                                                                                            Q                        2                                                              |                                                        ︸                    B                                                  ⁢                                                                  ⁢                                                                  ⁢                                                      |                                                                  USIZE                        ×                                                  (                                                                                                                                                      ⅆ                                  S                                                                                                  ⅆ                                  y                                                                                            ×                              Q                                                        -                                                                                                                            ⅆ                                  Q                                                                                                  ⅆ                                  y                                                                                            ×                              S                                                                                )                                                                                            Q                        2                                                              |                                                        ︸                    C                                                  ⁢                                                                  ⁢                                                      |                                                                  VSIZE                        ×                                                  (                                                                                                                                                      ⅆ                                  T                                                                                                  ⅆ                                  y                                                                                            ×                              Q                                                        -                                                                                                                            ⅆ                                  Q                                                                                                  ⅆ                                  y                                                                                            ×                              T                                                                                )                                                                                            Q                        2                                                              |                                                        ︸                    D                                                              )                                                          (        5        )            
FIG. 6 illustrates an example of the lod calculating apparatus 2 where an operation of equation (5) is carried out so as to calculate an lod.
A divider 11 divides 1 by inputted Q (operation of 1/Q) so as to output a resulting division to a multiplier 12, a multiplier 13, a circuit 21 (multipliers 32, 34), a circuit 22 (multipliers 42, 44), a circuit 24 (multipliers 52, 54), and a circuit 25 (multipliers 62, 64).
The multiplier 12 multiplies S by 1/Q so as to output a resulting product the circuit 21 (multiplier 35) and the circuit 22 (multiplier 45). The multiplier 13 multiplies T by 1/Q so as to output a resulting product the circuit 24 (multiplier 55) and the circuit 25 (multiplier 65).
The circuit 21 formed of a multiplier 31 through an absolute value detector 37 performs an operation of a portion corresponding to reference A (herein after referred to as portion A, similarly referred to for other portions) of equation (5), and the circuit 22 formed of a multiplier 41 through an absolute value detector 47 performs an operation of a portion C of equation (5), thus each outputting its operational result to a maximum value detector 23. The maximum value detector 23 detects the greater of a value from the circuit 21 (value of the portion A) and a value from the circuit 22 (value of the portion C) so as to output its operational result to a maximum value detector 27.
The circuit 24 formed of a multiplier 51 through an absolute value detector 57 performs an operation of a portion B of equation (5), and the circuit 25 formed of a multiplier 61 through an absolute value detector 67 performs an operation of a portion D of equation (5), thus each outputting its operational result to a maximum value detector 26. The maximum value detector 26 detects the greater of a value from the circuit 24 (value of the portion B) and a value from the circuit 25 (value of the portion D) so as to output its operational result to a maximum value detector 27.
The maximum value detector 27 detects the greater of a value from the maximum value detector 23 and a value from the maximum value detector 26 so as to outputting its result to a logarithmic operation unit 28. The logarithmic operation unit 28 performs an operation of a logarithm, having a base of 2, of the value from the maximum value detector 27 so as to output its operational result (operational result of the whole equation (5)) as an lod to the display buffer 4 (FIG. 4).
Then, in step S4, the texture mapping apparatus 1 calculates u data by dividing s data by q data and v data by dividing t data by q data for (s, t, q) of each pixel in order to obtain texture coordinate data (u, v).
In step S5, the texture mapping apparatus 1 obtains a texture address (U, V) based on the lod calculated by the lod calculating apparatus 2 and the texture coordinate data (u, v) which is a physical address of the texture buffer 3 so as to output the texture address to the texture buffer 3 and read texture data (R, G, B).
Then, in step S6, the texture mapping apparatus 1 writes, to the display buffer 4, pixel data obtained by treating the read texture data in step S5 with a predetermined process.
Subsequently, the operation is ended.
As described above, an access to the texture data corresponding to the lod out of a plurality of texture data stored in the texture buffer 3 each corresponding to the plurality of different reduction ratios is performed.